{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Active learning"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Active learning is a training regime, where the goal is to fit a model on the most discriminative samples. It is usually applied in situations where a limited amount of labeled data is available. In such a case, a human might be asked to annotate a sample. Doing this is expensive, so it's important to ask for labels for the most samples that will have the most impact.\n",
    "\n",
    "Online active learning is active learning done in a streaming fashion. Every time a prediction is made, an active learning strategy decides whether a label should be asked for or not. In case the strategy decides a yes, then the system could ask for a human to intervene. This is well summarized in the following schema from [Online Active Learning Methods for Fast Label-Efficient Spam Filtering](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=6fef6272cd72292e2f5a54d02d6e5352664e20cb)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div align=\"center\">\n",
    "    <img width=\"50%\" src=\"../img/online_active_learning.png\" />\n",
    "</div>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Online active learning"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "River's online active learning strategies are located in the `active` module. The latter contains wrapper models. These wrappers enrich the `predict_one` and `predict_proba_one` methods to include a boolean in the output.\n",
    "\n",
    "The returned boolean indicates whether or not a label should be asked for. In a production system, we could feed this to a web interface, and get the human to annotate the sample. Offline, we can simply use the label in the dataset.\n",
    "\n",
    "We'll implement this basic flow. We'll apply a TFIDF followed by logistic regression to a datasets of spam/ham received by SMS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-12-04T17:47:26.242700Z",
     "iopub.status.busy": "2023-12-04T17:47:26.242281Z",
     "iopub.status.idle": "2023-12-04T17:47:27.079931Z",
     "shell.execute_reply": "2023-12-04T17:47:27.079482Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Accuracy: 86.60%"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from river import active\n",
    "from river import datasets\n",
    "from river import feature_extraction\n",
    "from river import linear_model\n",
    "from river import metrics\n",
    "\n",
    "dataset = datasets.SMSSpam()\n",
    "metric = metrics.Accuracy()\n",
    "model = (\n",
    "    feature_extraction.TFIDF(on='body') |\n",
    "    linear_model.LogisticRegression()\n",
    ")\n",
    "model = active.EntropySampler(model, seed=42)\n",
    "\n",
    "n_samples_used = 0\n",
    "for x, y in dataset:\n",
    "    y_pred, ask = model.predict_one(x)\n",
    "    metric.update(y, y_pred)\n",
    "    if ask:\n",
    "        n_samples_used += 1\n",
    "        model.learn_one(x, y)\n",
    "    \n",
    "metric"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The performance is reasonable, even though all the dataset wasn't used for training. We can check how many samples were actually used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-12-04T17:47:27.094899Z",
     "iopub.status.busy": "2023-12-04T17:47:27.094694Z",
     "iopub.status.idle": "2023-12-04T17:47:27.105156Z",
     "shell.execute_reply": "2023-12-04T17:47:27.104823Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1921 / 5574 = 34.46%\n"
     ]
    }
   ],
   "source": [
    "print(f\"{n_samples_used} / {dataset.n_samples} = {n_samples_used / dataset.n_samples:.2%}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the above logic can be succinctly reproduced with the `progressive_val_score` function from the `evaluate` module. It recognises when an active learning model is provided, and will automatically display the number of samples used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-12-04T17:47:27.106972Z",
     "iopub.status.busy": "2023-12-04T17:47:27.106834Z",
     "iopub.status.idle": "2023-12-04T17:47:27.553244Z",
     "shell.execute_reply": "2023-12-04T17:47:27.552984Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,000] Accuracy: 84.80% – 661 samples used\n",
      "[2,000] Accuracy: 86.00% – 1,057 samples used\n",
      "[3,000] Accuracy: 86.37% – 1,339 samples used\n",
      "[4,000] Accuracy: 86.65% – 1,568 samples used\n",
      "[5,000] Accuracy: 86.54% – 1,790 samples used\n",
      "[5,574] Accuracy: 86.60% – 1,921 samples used\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Accuracy: 86.60%"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from river import evaluate\n",
    "\n",
    "evaluate.progressive_val_score(\n",
    "    dataset=dataset,\n",
    "    model=model.clone(),\n",
    "    metric=metric.clone(),\n",
    "    print_every=1000\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reduce training time"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Active learning is primarily used to label data in an efficient manner. However, in an online setting, active learning can also be used simply to speed up training. The point is that you can achieve a very good performance without training on an entire dataset. Active learning is a powerful way to decide which samples to train on."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Production considerations"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In production, you might want to deploy a system where humans may annotate samples queried by an active learning strategy. You have several options at your disposal, all of which go beyond the scope of River.\n",
    "\n",
    "The general idea is to have some kind of queue in which queried samples are fed into. Then you would have a user interface which displays the elements in the queue one-by-one. Each time a sample is labeled, the label would be used to update the model. You might have one or more threads/processes doing inference. You'll want to update the model in each one each time the model learns."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  },
  "vscode": {
   "interpreter": {
    "hash": "14b46bd212fa4dd89e3980db6ba7efbb9fe535833e1e483b914b71733e0a56d2"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
